Evolution of the Hypotheses Testing Approach in Intelligent Problem Solving Environments

In this paper, we compare different realizations of the hypotheses testing approach in the IPSEs (Intelligent Problem Solving Environments) ABSYNT, PETRI-HELP and MEDICUS and introduce the changes necessary to transfer the hypotheses testing approach to the real world domain of patent applications. Patent-IT is the first IPSE to overcome the limiting aspects of fixed specifications and a black box oracle. 1 Hypothesis Testing and the IPSE Approach The hypotheses testing approach is a core concept of what we call intelligent problem solving environments (IPSE) ([8], [9]) and also gives a key qualification having a beneficial influence on the student’s knowledge acquisition process. The learner acquires knowledge by actively exploring a domain, creating solution proposals for problems, testing hypotheses about their correctness, during which the system analyzes the proposals and provides help and explanations, making use of an oracle or an expert knowledge base. The IPSEs we developed initially had some limiting aspects such as fixed specifications and a black box oracle. We now present the IPSE Patent-IT, which does not exhibit these limitations. This has become necessary due to the rather demanding domain of inventions and patents. The novice should learn how to transform an inventive idea into a legal patent. The IPSE approach is psychologically based on the ISP-DL theory of knowledge acquisition and problem solving [8], which is influenced by theoretical assumptions of van Lehn [13], Newell [11], Anderson [1] and Gollwitzer [5]. It briefly states that new knowledge is acquired as a result of problem solving by applying weak heuristics in response to impasses. Furthermore, existing knowledge is optimized if applied successfully. The learner encounters four distinct problem solving phases namely deliberation, resulting in marking a goal as an intention, planning how to satisfy the intention, execution of the plan and evaluation of the result. Several design principles for IPSEs [8] could be drawn from the following assumptions: • The system should not interrupt the learner but offer help on demand. According to the theory, the learner will look for and appreciate help at an impasse. • Feedback and help information should be available on request at any time, taking the actual problem solving phase of the learner into account. • Help should be tailored to the learner’s pre-knowledge as much as possible. The best method to fulfill this requirement is to let the learner freely state hypotheses of the solution. 1.1 The Scope of our IPSEs so far and what is to come We have developed several IPSEs in variable application domains. The one common characteristic is that, the hypothesis testing process is fixed and hidden in a black box. The IPSEs ABSYNT and PETRI-HELP each define a closed world in which the learner explores a domain. MEDICUS defines the next step towards a non-closed world application, because real-world scenarios may act as a source for modeling. The modeling task is left to the learner, and MEDICUS is able to evaluate the equivalence of two different representations (specification, bayesian belief networks) of the real world scenario by an internal and fixed diagnostic process. Patent-IT is an IPSE, which supports the application of a patent by assessing critical aspects. The learner needs evaluation, judgement and argumentation skills to perform this task. He has to construct a model of his invention and is supported by the IPSE in his critiquing process. In this connection, the domain of patents serves as a kind of metadomain, which incorporates the hypothesis testing process. Neither a task specification nor the model are fixed in Patent-IT. The patentability of an invention is dependent on the state of the art. It is necessary that the model is not a derivative from the state of the art. This differs from previous IPSEs, where the model was to be equivalent to a specification or its logical conclusion. The state of the art for an invention is a real world concept. It is defined by a research in real world patent and literature databases and consists of a set of documents. By changing the model of the invention the state of the art also changes. This dynamic behavior of the domain makes a diagnostic process quite difficult and is realized in Patent-IT as a dialogue based cooperation of the system and the learner, which makes the process transparent and understandable for him. The critical dialogue depends on the modeled invention, the domain ontology known to the system, the results of the search for the state of the art and the user’s statements and arguments. By defending his invention, the learner may use his pre-knowledge as much as possible. In impasse situations where he needs support, Patent-IT is able to control the evaluation process, direct the learner and offer argumentative hints. Patent-IT is therefore the next consequent step on the evolutionary line of IPSEs. 1.2 ABSYNT (“Abstract Syntax Trees”) The ABSYNT [8] problem solving environment supports learners by offering help and proposals for functional programming in a graphical tree representation of pure LISP. The learner is given a fixed set of tasks. The programming task is internally represented as a symbolic goal, which triggers a set of transformation rules developed in the Munich CIP project ([2], [12]). In a diagnosic, hypotheses and help environment, the learner may visually state the hypothesis in a tree-like representation, that his solution proposal (or a boldly marked part of that proposal) to a programming task is correct. The system then analyzes this part of the solution proposal. One reason for the hypotheses testing approach is that, in programs, bugs cannot be absolutely localized. Therefore, the decision, which parts of a buggy solution proposal are to be kept, is left to the learner. This results in the system giving help and error feedback on the implementation level by synthesizing complete solutions, starting from the learner’s hypothesis. If the hypothesis is embeddable within a complete solution, the learner may ask for completion proposals. The hypotheses testing is a hidden process, based on a set of hundreds of diagnostic rules, defining a “goals -means-relation”, which analyzes and synthesizes several millions of solution proposals for 40 programming tasks. The rules, which are not shown to the user, generate complete solutions, recognize and complete incomplete proposals. The learner works in a closed world, defined by the tasks and the internal rules of the IPSE.